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Abstract 

We consider projection algorithms for solving (nonconvex) feasibility problems 
in Euclidean spaces. Of special interest are the Method of Alternating Projections 
(MAP) and the Douglas- Rachford or Averaged Alternating Reflection Algorithm (AAR). 
In the case of convex feasibility, firm nonexpansiveness of projection mappings is a 
global property that yields global convergence of MAP and for consistent problems 
AAR. Based on (e, (5)-regularity of sets developed by Bauschke, Luke, Phan and Wang 
in 2012, a relaxed local version of firm nonexpansiveness with respect to the inter- 
section is introduced for consistent feasibility problems. Together with a coercivity 
condition that relates to the regularity of the intersection, this yields local linear 
convergence of MAP for a wide class of nonconvex problems, and even local linear 
convergence of AAR in more limited nonconvex settings. 
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1 Introduction 

In the last decade there has been significant progress in the understanding of conver- 
gence of algorithms for solving generalized equations, and in particular those arising 
from variational problems such as minimization or maximization of functions, feasi- 
bility, variational inequalities and minimax problems. Early efforts focused on the 
proximal point algorithm [20 , 27] and notions of hypomonotonicity which is closely 
related to prox-regularity of functions [28] . Other works have focused on metric regu- 
larity and its refinements [2l[5]l21j . Proximal- type algorithms have been studied using 
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very different techniques in 0]. In a more limited context, the method of alternating 
projections (MAP) has been investigated in with the aim of formulating dual 

characterizations of regularity requirements for linear convergence via the normal cone 
and its variants. 

The framework we present here generalizes the tools for the analysis of fixed-point 
iterations of operators that violate the classical property of firm-nonexpansiveness in 
some quantifiable fashion. As such, our approach is more closely related to the ideas 
of [27] and hypomonotonicity through the resolvent mapping, however our applica- 
tion to MAP bears a direct resemblance to the more classical primal characteriza- 
tions of regularity described in [7] (in particular what they call "linear regularity"). 
There are also some parallels between what we call (S, e)-firm nonexpansiveness and 
e-Enlargements of maximal monotone operators [TT] , though this is beyond the scope 
of this paper. Our goal is to introduce the essential tools we make use of with a 
cursory treatment of the connections to other concepts in the literature, and to apply 
these tools to the MAP and AAR algorithms, comparing our results to the best known 
results at this time. 

We review the basic definitions and classical definitions and results below. In 
section [5] we introduce our relaxations of the notions of firm-nonexpansiveness and set 
regularity. Our main abstract result concerning fixed-point iterations of mappings that 
violate the classical firm-nonexpansive assumptions is in section [3] We specialize in 
subsequent subsections to MAP and the Douglas-Rachford algorithm, which we refer 
to as averaged alternating reflections (AAR). Our statement of linear convergence 
of MAP is as general as the results reported in [5], with more elementary proofs, 
although our estimates for the radius of convergence are more conservative. The 
results on local linear convergence of nonconvex instances of AAR to our knowledge 
are new and provide some evidence supporting the conjecture that, asymptotically, 
AAR converges more slowly (albeit still linearly) than simple alternating projections. 
Our estimates of the rate of convergence for both MAP and AAR are not optimal, and 
so a comparison between these two algorithms is not complete. We show, however, 
that strong regularity conditions on the set of fixed points are in fact necessary for 
local linear convergence of the AAR algorithm, in contrast to MAP where the same 
conditions are sufficient, but not necessary [8l|9]. 

1.1 Basics/Notation 

E is a Euclidean space. We denote the unit ball centered at the origin by B and the 
ball of radius 8 centered at x 6 E by Mg (x) . When the 6-ball is centered at the origin 
we write Mg. The Notation "=f' indicates that this mapping in general is multi-valued. 

Definition 1. Let f2 C E be nonempty, x € E. The distance of x to £1 is defined by 



Definition 2 (projectors/reflectors). Let f2 C E be nonempty and x G E. The 
(possibly empty) set of all best approximation points from x to Q denoted Pn(x) (or 
Pqx), is given by 



d (x, ft) :— min \\x — y\\ . 



(1.1) 



Pn(x) :={ye ft| \\x - y\\ = d(x,Q)} . 



(1.2) 
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The mapping Pq =4 fl is called the metric projector, or projector, onto fl. We call an 
element of this set a projection. The reflector Rq : E =4 E to the set is defined as 

R n x := 2P n x - x, (1.3) 

for all x E E. 

Since we are on a Euclidean space E convexity and closedness of a subset C C E is 
sufficient for the projector (respectively the reflector) to be single valued. Closedness 
of a set Q suffices for the set fi being proximinal, i.e. Pqx ^ for all i€E. 

Definition 3 (Method of Alternating Projections). For two sets A,Bc~Ewe call 
the mapping 

T MAP x = PaPbx (1.4) 

the Method of Alternating Projections operator. We call the MAP algorithm, or simply 
MAP, the corresponding Picard iteration, 

x-n+i G Tmapx„,, n = 0,1,2,... (1.5) 

for x given. 

Definition 4 (Averaged Alternating Reflections/Douglas Rachford). For two sets 
A,BcE we call the mapping 

T AAR x=^(R A R B x + x) (1.6) 

the Averaged Alternating Reflection (AAR) operator. We call the AAR algorithm, or 
simply AAR, the corresponding Picard iteration, 

x n+1 GT AAR x n , n = 0,1,2,... (1.7) 

for xq given. 

Example 5. The following easy examples will appear throughout this work and serve 
to illustrate the regularity concepts we introduce and the convergence behavior of the 
algorithms under consideration. 

(i) Two lines in K 2 : 

A = {(x!,x 2 ) e M 2 | x 2 = 0} c K 2 
B = {(x 1 ,x 2 ) € M 2 | X! = x 2 } C M 2 . 

We will see that MAP and AAR converge with a linear rate to the intersection. 

(ii) Two lines in K 3 : 

A = {(x ll x 2 ,x 3 ) eS 3 \x 2 = 0,x 3 = 0} CR 3 
B ={(xi,x 2 ,x 3 ) S R 3 | x 1 = x 2 ,x 3 = 0} C M 3 . 

After the first iteration step MAP shows exactly the same convergence behavior 
as in the first example. AAR does not converge. All iterates stay in a plane 
parallel to A + B. 
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(iii) A line and a ball intersecting in one point: 

A = {(xi,x 2 ) el 2 |i 2 = 0}ct 2 

B = {(xi,x 2 ) G K 2 | x\ + (x 2 - l) 2 < 1} . 

MAP converges to the intersection, but convergence is arbitrarily slow, the slower 
the iterates are to the intersection. AAR has fixed points that lie outside the 
intersection. 

(iv) A line and a non-calm set: 

A = {( Xl ,x 2 ) £K 2 | xi = 0} 

B ={(xi,x 2 ) e M 2 | x 2 = xi sin(l/a;i)} . 

(v) A circle and a line: 

A = {(x u X2) G K 2 | x 2 = V2/2} c K 2 
B={(x u x 2 ) e R 2 I x\ + x 2 = 1} . 

This example is of our particular interest, since it is a simple model case of the 
phase retrieval problem. So far the only direct nonconvex convergence results for 
AAR are related to this model case, see [TUTU]. Local convergence of MAP is 
covered by [HUH] as well as by the results in this work. 

□ 

1.2 Classical results for (firmly) nonexpansive mappings 

To begin, we recall (firmly) nonexpansive mappings and their natural association with 
projectors and reflectors on convex sets. We later extend this notion to nonconvex 
settings where the algorithms involve set- valued mappings. 

Definition 6. Let Q C E be nonempty. T : Q — > E is called nonexpansive, if 

\\Tx-Ty\\<\\x-y\\ (1.8) 

holds for all x,y £ £1. 

T : O — > E is called firmly nonexpansive, if 

\\Tx - Ty\\ 2 + \\(Id -T)x- (Id -T)y\\ 2 <\\x~y\\ 2 (1.9) 

holds for all x,y £ f2. 

Lemma 7 (Proposition 4.2 [5 ]). Let C E be nonempty and let T : 51 —> E . The 
following are equivalent 

(i) T is firmly nonexpansive on f2 

(ii) IT — Id is nonexpansive on 

(iii) \\Tx — Ty\\ 2 < (Tx — Ty, x — y) for all x,y E Q 
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Theorem 8 (best approximation property - convex case). Let C C E be nonempty 
and convex, x G E and x G C. x is the best approximation point x = Pc(x) if and 
only if 

(x-x,y-x) < for ally G C. (1.10) 
// C is a affine subspace then (jl.lOjl holds with equality. 

Proof. For (|1.10[) see Theorem 3.14 of [3], while equality follows from Corollary 3.20 
of the same. □ 

Theorem 9 ((firm) nonexpansiveness of projectors/reflectors). Let C be a closed, 
nonempty and convex set. The projector Pq : E — > E is a firmly nonexpansive mapping 
and hence the reflector Rq is nonexpansive. If, in addition, C is an affine subspace 
then following conditions hold. 

(i) Pc is firmly nonexpansive with equality, i.e. 

\\Pcx - Pcv\\ 2 + || (Id -P c )x - (Id -P c )y\\ 2 = \\x - yf , (1.11) 

for all x G E. 

(ii) For all x G E 

\\Rcx-c\\ = \\x-c\\ (1.12) 

holds for all c G C . 

Proof. For the first part of the statement see [121 Theorems 4.1 and 5.5], [121 Chapter 
12], [17l Propositions 3.5 and 11.2] and [30l Lemma 1.1]. The well-known refinement 
for affine subspaces follows by a routine application of the definitions and Theorem 

m □ 



2 (S, e)-firm nonexpansiveness 



Up to this point, the results have concerned only convex sets, and hence the projector 
and related algorithms have all been single-valued. In what follows, we generalize to 
nonconvex sets and therefore allow multi-valuedness of the projectors. 

Lemma 10. Let A, B C E be nonempty and closed. Let x G E. For any element 
x + G Taarx there is a point x G RaRb such that x+ — \{x + x). Moreover, Taar 
satisfies the following properties. 

ft) 

\\x + y + \\ 2 + \\(x - x + ) "(2/-y+)|| 2 = i||x-y|| 2 + i||£-y|| 2 (2.1) 

where x and y are elements o/E ; x+ and y+ are elements o}Taarx and TaarV 
respectively, and x G RaRbX and y G RaRbV are the corresponding points 
satisfying x+ — ^(x + x) and y+ — \ {y + y). 

(ii) For all x G E 



Taarx = {P A (2z-x)-z + x I 2G P B x} . (2.2) 
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Proof. By Definition @] 



x+ £ Taarx 
x + £ ^(RaRbx + x) 
2x+ — x £ RaRbx. 



Defining x = 2x + — x yields X-\- = j(x + x), where x £ RaRbx. 

(i) For x+ £ Taarx (respectively y+ £ Taaru) choose x £ RaRbx (respectively y) 
such that x+ — (x + x)/2 (respectively y+). Then 

\\x+ - y+\\ 2 + \\(x~x+) -(y- y+)\\ 2 



-JJ 



=2 ll a; -2/ll 2 + 2 ll^ _ yll 2 

1,. - ,1 
+ -j\x- y,x- y) - - 

~\\\x - y\\ 2 + ]^\\x - y\\ 2 ■ 



(x-y,x-y) 



(ii) This follows easily from the definitions: 



Taarx 



- (R A v + x) 



v £ Rbx 

z £ Pbx 



-{R a {2z-x)+x) 



(2P A (2z-x) - z + x) 



where v — 2z — x, z £ Pbx 



Z £ PnX 



{Pa{2z -x)-z + x\ z£ P b x} . 



□ 



Remark 11. In the case where A and B are convex, then as a consequence of Lemma 
HOl i) and the fact that the reflector i?n onto a convex set f2 is nonexpansive, we recover 
the well-known fact that Taar is firmly nonexpansive and (12.1[) reduces to 



(2.3) 



\Taarx - T A ARy\\ 2 + II (W -Taar) x- (Id -T AAR ) y\\ 2 

1 2 1 2 

= 2\\ x ~y\\ + -\\RaRbx - R a Rbv\\ , 



while (|2.2p reduces to 



Taarx 



PaRbx - Pbx. 



(2.4) 



We define next an analog to firm nonexpansiveness in the nonconvex case with 
respect to a set S. 

Definition 12 ((S 1 , e)-(firmly-)nonexpansive mappings). Let D and S be nonempty 
subsets of E and let T be a (multi-valued) mapping from D to E. 



6 



i) T is called (S, e)-nonexpansive on D if 



\\x+ — ar+|| < y/T+e \\x — x\\ 
\/x £ D, \/x £ S, \/x+ £ Tx, Vx+ £ Tx. 



(2.5) 



If \2.5\ holds with e — £/ien we saj/ i/iai T is ^-nonexpansive on D. 
ii) T is called (S, e)-firmly nonexpansive on D if 

\\ x + — x+\\ 2 + \\(x — x+) — (x — x + )\\ 2 < (1 + e) || x — x\\ 
Vx £ D, \fx £ S, \/x + £ Tx, \/x+ £ Tx. 



(2.6) 



If \2.6\ holds with e = then we say that T is iS-firmly nonexpansive on D. 

Note that, as with (firmly) nonexpansive mappings, the mapping T need not be 
a self-mapping from D to itself. In the special case where S = FixT, mappings sat- 
isfying (|2.5[) are also called quasi- (firmly- Jnonexpansive [5]. Quasi- nonexpansiveness 
is a restriction of another well-known concept, Fejer monotonicity, to FixT. Equa- 
tion (|2.6[) is a relaxed version of firm nonexpansiveness (ll.9[) . The aim of this work 
is to expand the theory for projection methods (and in particular MAP and AAR) 
to the setting where one (or more) of the sets are nonconvex. The classical (firmly) 
nonexpansive operator on D is (D, O)-(firmly) nonexpansive on D. 

Analogous to the relation between firmly nonexpansive mappings and averaged 
mappings (see [5J Chapter 4] and references therein) we have the following relation- 
ship between (S, e)-firmly nonexpansive mappings and their 1/2-averaged companion 
mapping. 

Lemma 13 (1/2— averaged mappings). Let D,ScE be nonempty and T : D =4 E. 
The following are equivalent 

(i) T is {S,e)-fi,rmly nonexpansive on D. 

(ii) There is a mapping T that is (S, 2s) -nonexpansive on D with 



Proof. For x £ D define T : D =} E by Tx := (2Tx — x) and choose x + £ Tx. Observe 




(2.7) 




< \ \\x-x\\ 2 + \{\ + 2e)\\x-x\ 
= {l + e)\\x-x\\ 2 



where the inequality holds if and only if T is 2e— quasinonexpansive. 



□ 
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2.1 Regularity of Sets 

To achieve property (I2.6p for the projector and the AAR operator, we have to require 
some properties to the sets A and B upon which we project. These are developed 
next. 

Definition 14 (Prox-regularity). A nonempty (locally) closed set Cl C E is prox- 
regular at a point x £ Cl if the projector Pq is single-valued around x. 

What we take as the definition of prox-regularity actually follows from the equiv- 
alence of prox-regularity of sets as defined in 28, Definition 1.1] and the single- 
valuedness of the projection operator on neighborhoods of the set [28, Theorem 1.3]. 

Definition 15 (normal cones). The Frechet normal cone Nq(x) and the limiting 
normal cone Nn(x) to a set Cl C E at a point x £ Cl are 

Nn(x) := \v £ E : limsup ^"jf < 1 , (2.8) 

^Vh (x) := lim sup Nq (x) , (2.9) 

n _ 

The construction of the limiting normal cone goes back to Mordukhovich (see [291 
Chap. 6 Commentary]). 

Proposition 16 (Mordukhovich). The limiting normal cone or Mordukhovich normal 
cone is the smallest cone satisfying the two properties 

Lye P n (x) ^x-y£ N n (y), 

2. for any sequence Xi — > x in Cl any limit of a sequence of normals Vi £ Nq(xv) 
must lie in Nq(x). 

Definition 17 (Clarke regularity). A nonempty (locally) closed set CI C E is Clarke 
regular at a point x £ CI if the Frechet normal cone and the limiting normal cone 
coincide, i.e. Nq(x) — N^(x). 

Definition 18 ((e, J)-(sub)regularity). 

i) A nonempty set Cl C E is (e, (J)-subregular at x with respect to S C E, if there 
exists e > 0, S > 0, and 

(v,z-y)<e\\v\\\\z-y\\ (2.10) 

holds for all y £ M$(x) fl Cl, z £ S n B^x), v £ Nn(y). We simply say Cl is 
(e, 6 )-subregular atx if S — {x}. 

ii) If S — Cl in i) then we say that the set Cl is (e, (5)-regular at x. 

Hi) If for all e > there exists a S > such that (|2.10[) holds for all y,z £ Mg(x) PtCl 
and v £ Nn(y), then Cl is said to be super- regular. 

The definition of (e, (5)-regularity was introduced in |5J Definition 9.1] and is a 
generalization of the notion of super-regularity introduced in [531 Definition 4.3]. More 
details to (e, 5)-regularity can be seen in [S]. Of particular interest is the following. 

Proposition 19 (Prox-regular implies super-regular, Proposition 4.9 of [23] )• If a 
closed set Cl C E is prox-regular at a point in Cl, then it is super-regular at that point. 
If a closed set Cl C E is super-regular at a point in Cl, then it is Clarke regular at that 
point. 
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Superrcgularity is something between Clarke regularity and amenability or prox- 
regularity. (e, <5)-regularity is weaker still than Clarke regularity (and hence super- 
regularity) as the next example shows. 

Remark 20. (e, <5)-regularity does not imply Clarke regularity 
Proof. 

X 2 < —x\ X\ < 



il := | 



x 2 <0 xi > 



For any x_ G d-Sl := {(x\, x 2 )\x 2 = —x\, x 2 < 0} and x + G <9 + := {(xi,0)|xi > 0} 
one has 



N n (x-)={(\,\)\\£ 
N n (x+)={(0,\)\\eR+} 

which implies Afo(0) = N n (x-) U N n (x + ). Since Afo(0) = {0} the set Q is not Clarke 
regular at 0. Define v_ = (y/2/2,V2/2) G N a (x-), v + = (0, 1) G N n {x + ) and note 

(^_,0 — x_) = and (^ + ,0 — x + ) = 

to show that VL is (0, oo)— subregular at 0. By the use of these two inequalities one 
now has 

(v-,x+ -x-) = (u-,x+) < V2/2\\x+\\ 
(u+,x--x+) = (u+,x-) < V2/2\\x-\\ 

and hence fi is (^2, oo)-regular. □ 
2.2 Projectors and Reflectors 

We arc now ready to apply the above definitions in order to establish (S, e)(firm)- 
nonexpansiveness of projectors and reflectors. 

Theorem 21 (projectors and reflectors onto (s, <5)-subregular sets). Let SI C E be 
nonempty closed and (e, S )-subregular at each point igScSJ. 

(i) The projector is {S, S\) -nonexpansive on M${S) where S\ = 2e + e 2 : at each ieS 



\\x+ -x\\ < y/l + ex \\x-x\\ \fx G M s (x), x+ e Pnx. (2.11) 

=(i+s) 

(ii) The projector is (S , s 2 ) -firmly nonexpansive on Ms(S), where s 2 = 2s + 2s 2 : at 
each SgS 

\\x + - x\\ 2 + \\x - x + \\ 2 < (1 +s 2 ) \\x- x\\ 2 Vx G M§(x), x + G Pqx. (2.12) 

(Hi) The reflector Rq is (S, is) -nonexpansive on Ms(S), where S3 = As + As 2 : at each 
x G S ' 

\\x+ - x\\ < y/l + 4e(l + e) \\x - x\\ Va; e M s (x), x+ e Rqx. (2.13) 
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Proof. (i) The projector is nonempty by closedness. Then by the definition of (e, <$)- 
subregularity and the Cauchy-Schwarz inequality, for each x E S 

\\x + — x|| 2 =(x — x, x+ — x) + (x+ — x, x+ — x) 

< \\x — x\\ ||x + — x| + e \ \x — x+\\ ||x + — x\\ 

<(1 + e) \\x ~ x\\ \\x+ - x\\ Vx G M s (x), x + G Pox. 

Thus at each x E S 

\\x+-x\\ < (1 + e) \\x - x\\ = ^(l + e) 2 \\x-x\\ = y/l + {2s + s 2 ) \\x - x\\ 

for all x G Ms(x), as claimed. 

(ii) By the definition of (e, (J)-subregularity (|2.10l) . at each x G S 

\\x + - x\\ 2 + \\x - x + \\ 2 
= \\x+ — x\\ 2 + || x — x + x — x + 1| 2 

= \ \x + — x\\ 2 + 1 1 a; — x\\ 2 + 2(x — x. x — x + ) + ||.t + — x\\ 2 

=2 \ \x+ — x\\ 2 + \\x — x\\ 2 + 2 (x+ — x,x — x + ) +2 (x — x + , x — x + ) 

s v / S ' 

— — aT|| <e||x— x+ \\ — x\\ 

< \\x- x\\ 2 + 2e\\x + - x\\ \\x- x + \\ Vx G M s (x), x + G Pq,x. (2.14) 

By definition ||x — x+|| < ||x — a?||. Combining (|2.14l) and equation (|2.1ip yields 
at each x G S 

\\x+ -x\\ 2 + ||x-x + || 2 < (1 + 2e(l +e)) ||x-x|| 2 Vx G M s (x), x+ G P n x. 

(iii) For each x G by JnJ the projector is ({x},2e + 2£ 2 )-firmly nonexpansive on 
B,5(x), and so by Lemma [T51 fn)) Rq is ({x}, 4e + 4e 2 )-nonexpansive there. Since 
x G S is arbitrary this proves the result. 

□ 

Note that i\ < £2 (s > 0) in the above theorem, in other words, the degree to which 
classical firm nonexpansiveness is violated is greater than the degree to which classical 
nonexpansiveness is violated. This is as one would expect since firm nonexpansiveness 
is a stronger property than nonexpansiveness. 

We can now characterize the degree to which the AAR operator violates firm-non- 
expansiveness on neighborhoods of (e, 5)-subregular sets. 

Theorem 22 ((5, e)-firm nonexpansiveness of Taar)- Let A,BcEbe closed and 
nonempty. Let A and B be {ea, 5)- and {eb, 5)-subregular respectively at each ieSc 
Ap\B. The AAR operator Taar : E =^ E defined by (|1.6|) is (S,e)-firm nonexpansive 
on Mg(S) where e — 2ea{^ + £a) + 2e_e(l + £b) + &£a{^ + £a)£b{^- + £b), that is 

ateachxES, ||x + — x|| 2 + ||x — x + || 2 < (1 + e) ||x — x|| 2 (2-15) 
Vx G B s (x), x+ eTaarx. (2.16) 

Proof. Choose any x G S. Note that RaRbx = Rb% = x since x G An B. For 
x + G Taarx choose x G RaRbx, such that x+ = (x + x)/2. Then by Theorem 121 rfiTT|) 
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we have for all x £ Ms (x) 



\\ x — x\\ <yl + 4^(1 + ea) \\v ~ x\\ , for some y £ Rbx 



<y/l + 4e A (l + e A ) y/l + 4e s (l + e B ) \\x - x\ 



=Vl + 2e\\x-x\ 



The result then follows from Lemma [13] fU . 



□ 



If one of the sets above is convex, say B for instance, the constant e simplifies to 
£ = 2e^(1 + ea) since B is (0, oo)-subregular at x. If both sets are convex the we 
recover [TU] iii) . 

3 Linear Convergence of Iterated (5, £)-firmly non- 
expansive Operators 

Our goal in this section is to establish the weakest conditions we can (at the moment) 
under which the MAP and AAR algorithms converge locally linearly. The notions 
of regularity developed in the previous section are necessary, but not sufficient. In 
addition to regularity of the operators, we need regularity of the fixed point sets of 
the operators. This is developed next. 

Despite its simplicity, the following Lemma is one of our fundamental tools. 

Lemma 23. Let D, S C E, T : D =t E and U C D \ FixT. If 

(a) T is (S , e) -firmly nonexpansive onflcE and 

(b) for some point xeScE 

||a; — x + — (x — x + )\\ > A \\x — x\\ V x + 6 Tx, Vi+e Tx, \fx e U, (3.1) 
then T is ({x},£ — X 2 ) -nonexpansive on U: 

\\x+ - x+\\ <^/(l + £ - A 2 ) ||x - x|| V x+ e Tx, V x+ e Tx, Vx g U. (3.2) 
In particular, for S C Fix T, 



Proof. Choose any x+ £ Tx and x+ £ Tx. Combine equations (|3.ip and (|2.6p to get 



Based on (e, 5)-(sub-)regularity it has already been shown that both MAP and AAR 
are (S, e)-firmly nonexpansive if S C AnB. What remains to be shown is the existence 
of a point x such that (|3.ip is fulfilled. 

In what follows we determine conditions on the intersection A n B to guarantee 
that the coercivity condition (|3 . 1 [) holds. We emphasize that the properties (a) and 
(b) in Lemma [231 are independent. Example [5] (pill) describes the intersection of two 



\\x+ - x\\ <y/{l + e - A 2 ) ||x - x\\ Vx+£Tx,Vx£U. 



(3.3) 



||x+ - x + \\ 2 + (A || x — x\\) 2 < 
||x + — x + || 2 + ||x — x + — (x — x + )|| 2 <(1 + e) ||x — x| 



for all x £ U , which immediately yields (|3.2j) . 



□ 
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convex sets, so (a) holds, but as we show below (b) fails. As a result the MAP 
iterates, for example, converge arbitrarily slowly. On the other hand, in Example [S] 
(|rv)) describes two sets whose intersection is "well behaved" even though one of the 
sets is not regular. Combining both properties (a) and (b) in the end (and having a 
proper balance between them) yields convergence. 

3.1 Regularity of Intersections 

We define two notions of regularity of the intersection of sets. The first one, which we 
call strong regularity, has many different names in the literature, among them linear 
regularity [24 . We will use the term linear regularity to denote a weaker regularity 
than strong linearity. Both terms "strong" and "linear" are overused in the literature 
but we have attempted, at the risk of some confusion, to conform to the usage that 
best indicates the heritage of the concepts. 

Definition 24 (strong regularity, Kruger [22]). The collection of sets f2i, f^, • • • > 
is strongly regular at x if there exists an a > and a 5 > such that 

(n^^-^-a^ni^e (3.4) 

for all p G (0, 6], uii G £1; n M$(x), ai G B ap , i = 1,2, . . . ,m. 

Theorem 25 (strongly regular intersection, Theorem 1 [23]). A family of m closed, 
nonempty sets Qi,Q,2, ■ ■ . , £l m has strongly regular intersection atx£ CiJ^i^lj if and 
only if there exists a k > and a 6 > such that 

d(x,n'jL 1 (Slj - Xj)) < k max d (x + a*, fii) , Va; G Ms(x), (3.5) 

J i=l,...,m 

for all x G Ms(x), Xi G B<5, i = 1, . . . ,m. 

Theorem 26 (Theorem 1 [22]). A family of closed sets fii, f22> • • • i &m ^ E has 
strongly regular intersection (|3.4p at a point x G PljOj, if the only solution to the 
system 

m 

Vi — 0, with Vi G Nn i (x) for i = 1, 2, . . . , m 

i=l 

is i?j = /or i = 1, 2, ... , m. For two sets Oi, C E </iis can 6e written as 

N ni ($n-Nn a (TB) = {0}, (3.6) 
anrf is equivalent to the previous Definition p.4p . 

Definition 27 (linear regularity). A family of closed, nonempty sets Q±, Q2, ■ ■ ■ , 
has locally linearly regular intersection at x G f^J^i^j if there exists a k > and a 
5 > smc/i that 

d(x,n" l =l ttj) < k max d(a:,fif), Va; G M s (x). (3.7) 

J z— l,...,m 

If (|3.7p ZioMs /or any (5 > </ie intersection is called linearly regular. The infimum 
over all k such that (13.71) holds is called regularity modulus. 

Since (|3.7I) is (|3.5I) with Xj = for all j = 1,2, ... ,m, it is clear that strong 
regularity implies local linear regularity and is indeed a much more restrictive notion 
than local linear regularity. 
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Remark 28. What we are calling local linear regularity at x has appeared in various 
forms elsewhere. See for instance [THl Proposition 4] , [251 Section 3] , and [23} Equation 
(15)]. Compare this to (bounded) linear regularity defined in [JJ Definition 5.6]. Also 
compare this to the basic constraint qualification for sets in |25l Definition 3.2] and 
strong regularity of the intersection in |23l Proposition 2] , also called linear regularity 
in [H]. 

Remark 29. Based on strong regularity (more specific characterization f|3 .6j) ^1 Lewis, 
Luke and Malick proved local linear convergence of MAP in the nonconvex setting, 
where both sets A,B are closed and one of the sets is super-regular [23] ■ This was 
refined later in [5] . The proof of convergence that will be given in this work is different 
from the one used in [9"ll24| and more related to the one in [7] . Convergence is achieved 
using (local) linear regularity (|3.7[) . which is described in [T4j Theorem 4.5] as the pre- 
cise property equivalent to uniform linear convergence of the CPA (Cyclic projections 
algorithms) . However the rate of convergence achieved by the use of linear regularity 
is not optimal, while the one in [H][23] is in some instances. An adequate description 
of the relation between the direct/primal techniques used here and the dual approach 
used in [9][24] remains open. 

Theorem 30 (linear regularity of intersections of convex cones). Let Sl-y, O2, • • • , 
be a family of closed, nonempty, convex cones. The following statements are equivalent 

(i) The intersection is locally linearly regular atx 

(ii) The intersection is linearly regular atx 

Proof. [7J Proposition 5.9 □ 

Example 31 (Example [5] revisited.). The intersection in example [5] Q at is strongly 
regular (c = y/2/2) and linearly regular (k — v2/4). The same example embedded 
in a higher-dimensional space is still linearly regular, but looses its strong regularity. 
This can be seen by shifting one of the sets in example [5] in ^-direction, as this 
renders the intersection empty. This shows that linear regularity does not imply strong 
regularity. The intersection in example (pUl) is neither strongly regular nor linearly 
regular. In each of the examples [5] (EvJ) and (jvj) one of the sets is nonconvex, but the 
intersections are still well-behaved. Observe that in dry]) shifting any of the sets does 
not render the intersection empty and so the intersection is strongly (locally) regular. 
Although this already implies linear regularity in (jrv)) . we state the following to show 
linear regularity more directly. Note that since 



The next definitions, while not necessary for our purposes here, provide a bridge 
between the above notions of regularity and the approaches of [2,3, 20 , 21 , 27 . 

Definition 32 ((strong) metric (sub)-regularity) . 

(i) The mapping $ : E =4 Y is called metrically regular at x for y if there is a 
constant k > together with neighborhood U ofx and Vofy such that 



The regularity modulus reg$(x|y) is the infimum of those constants k > such 
that (|3~E)) holds. 




by linear regularity of A n B linear regularity of A n B follows. 



□ 




(3.8) 
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(ii) The mapping $ : E =; Y m called metrically subregular at x for y if (x,y) G 
gph<f> and there is a constant k > and neighborhoods Uofx and Vofy such 
that 

d (x, < nd (y, $(x) n V) for all x€U. (3.9) 

TTie subregularity modulus subreg$(57|y) is the infimum of those constants k > 
such that (13.91) holds. 

(Hi) The mapping $ : E =4 Y is strongly metrically subregular 3; /or y z/ (x, y) € 
gph<i> and there is a constant 7 > along with neighborhoods Uofx and Vofy 
such that 

\\x-x\\<Kd(y,$(x)nV) for all x e 17. (3.10) 

Proposition 33 (Metric regularity and strong regularity). Consider the set-valued 
mapping $ : E =$ E m 

$(x) = (fii — x) X • • • X (fi m — x) 

We have the following characterizations: 

(i) $ is metrically regular at X for if and only if the intersection HiOi is strongly 
regular at x . 

(ii) $ is metrically subregular at x for if and only if the intersection Hi f2j is locally 
linearly regular. 

(Hi) $ is strongly metrically subregular at x for if and only if the intersection is 
locally linearly regular and {x} — (n,fij) (1 85(5) for all S > small enough. 

Proof. The first statement was proved in [241 . 
To prove the second statement note that 

m rn 

((yi, y m )) = p| {x\yj G - x} = f| - 

Hence 

rn 

$- i ((o,...,o))=n% 
3=1 

and (5,0) G gph<f> is equivalent to x G Hjli^j- Also by norm equivalence, the 
existence of Ms (x) such that 

d (x, ((0, . . . , 0))) < n x d (0, $(x) n V) for all x € £7 

is equivalent to the existence of K2 > (different than the Ki above) such that 

d(x,Q) < K2 max d(x, Q,-) for all a; G Ba(x) . 

j=l,...,m 

As this is by definition local linear regularity of (Oi, . . . , f2 m ) at x, the proof is com- 
plete. 

The third property then follows by basic properties of strong metric subregularity, 
that is that strong metric subregularity at a point x for y is equivalent to metric 
subregularity $ at x for y and x being a isolated point of □ 
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3.2 Linear Convergence of MAP 

In the case of the MAP operator, the connection between local linear regularity of the 
intersection of sets and the coercivity of the operator with respect to the intersection 
is natural, as the next result shows. 

Proposition 34 (coercivity of the projector). Let A,B be nonempty and closed sub- 
sets of E and let x\ € A and X2 € Pbx%. If S = An B is locally linearly regular at 
x 6 Psx\ then 

\\x2-xi\\ >7ll*i-s|| (3-11) 
where 7 = 1/k and n is the regularity modulus. 
Proof. Since x\ € A one has 

\\x2 - xi\\ =d(xi,B) 

= max i^d (xi, B) , d (x\, A) 

>jd(xi,S) 
=7||xi -x\\ 

where the inequality follows by Definition [57] (local linear regularity at x) . □ 

Theorem 35 (Projections onto a (e, <5)-subregular set). Let A,B be nonempty and 
closed subsets o/E and let x\ £ A and xi € PbX\. If 

(a) S = A(~] B is locally linearly regular atx£ PsX\ and 

(b) B is (e, 5)-subregular at x with 5 > d (x±, S) 
then 



d(x 2 ,S) < ^l + £--/ 2 d(x u S), (3.12) 

where 7 = 1/k with k the regularity modulus and e = 2e + 2e 2 . 

Proof. By Proposition [341 we have that 

\\x2 -xi\\ >j\\xi -x\\ . 

If x\ e S then x\ = X2 € S and the result is trivial. Assume then that X\ ^ B. Since 
X\ G A is arbitrary, the coercivity condition on the intersection in (|3.ip is satisfied 
at least on A \ S C E \ FixPg. Since B is (e, <5)-subregular at x, by Theorem [211 
the projector Pb is ({x}, 2e + 2e 2 )-firmly nonexpansive on Mg(x). In particular, since 
5 > d(x\,S) Lemma [2^1 then applies to yield 



\\x 2 - x\\ <vl + e - 7 2 || xi - x\\ . 

Since x € PsX\ we have d(x%, S) = \\xi — x\\, and by definition d(x2, S) < \\x2 — x\\. 
This completes the proof. □ 

The following corollaries are direct consequences of Theorem [35J 
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Corollary 36 (Projections onto a convex set [18] ). Let A, B be nonempty and closed 
subsets of E and let x\ G A and x 2 G Pb%i ■ If 

(a) S = A(~] B is locally linearly regular atx*E Ps%i end 

(b) B is convex 
then 



d(x 2 ,S) < Vl-l 2 d( Xl ,S), (3.13) 
where 7 = 1/k with k the regularity modulus. 

Proof. Since B is convex it is (0, +oo)-regular hence e = in Theorem 1351 □ 

Corollary 37 (linear convergence of MAP). Let A, B be closed nonempty subsets of 
E with locally linearly regular intersection atx<E S := Af] B. For any xq G M$(x), 
generate the sequence {j n }n6N by 

X2n + 1 € PAZ2nand X 2n +2 G PbX2u+1 (Vn = 0, 1, 2, . . . ). (3-14) 

(a) If A and B are (e,6)—subregular w.r.t. S, then 

d (x 2 „+ 2 , S) < (1 - 7 2 + e)d (x 2 „, S) . 

(b) If A is (e , 6)— subregular w.r.t. S and B is convex, then 



d (x 2n+2 ,S) < y/1 - 7 2 + ey/1 - 7 2 d (X 2n , S) . 
(C,) If A and B are convex, then 

d (x 2n +2,S) < (1 - 7 2 ¥ (%2n,S) . 

3.3 Linear Convergence of AAR 

We now turn to the AAR algorithm. This algorithm is notoriously difficult to analyze 
and our results reflect this in considerably more circumscribed conditions than are 
required for the MAP algorithm. Nevertheless, to our knowledge the following conver- 
gence results are the most general to date. The first result gives sufficient conditions 
for the coercivity condition [53] (b) to hold. 

Lemma 38. Let the closed subsets A, B o/E have locally linearly regular intersection 
at x G S = An B with regularity modulus k > and radius of regularity S in (|3.7p . 
Suppose further that B is a subspace and that for some constant c G (0, 1) the following 
condition holds: 

x G B s (x), y G P B x, z G P A {2y - x) and " | jj^j ^ J J (u,v) > -c. 

(3.15) 

Then Taar. fulfills the coercivity condition 



\x-x+\\ > %/1 ° d(x,S) Viet/ (3.16) 



K 

where x + G Taarx and U = {x G B|5(x)|Pa.Rbx C B,$(x)}. 
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Proof. In what follows we will use the notation Rgx for 2y — x with y £ Pbx which 
is unambiguous, if a slight abuse of notation, since B is convex. We will use (|3.15[) to 
show that for all x £ M$(x) fixed with y £ Pbx and z £ PaRbx 



\x-x+f = \\z-yf 



> (1-c) \\z- R B x\\ 2 + \\R B x-y\\ 2 



(3.17) 



We will then show that, for all x £ U fixed with y £ Pbcc and z £ PaRbx 

\\z — Rbx\\ 2 + \\Rbx — y\\ 2 > -Kd (Rbx, S) 2 . 

hi 

Combining inequalities (13. 17)) and (|3.18j) yields 



(3.18) 



\\x-x+\\ 2 >^—^d(R B x,S) 2 VxeU. 

Let x € Ps(Rbx) and note that d(RBX,S) = \\Rbx — x\\. Since B is a subspace, by 
(|1.12[) one has d (Rbx, S) = H-Rbx — x|| = ||x — 

Moreover, ||x — x\\ > minj, 6 5 \\x — y\\ = d (x, S), hence 



||g-g + || > A/1 ° d(x,S) VxeU 



as claimed. 

What remains is to prove (|3.17[) and (|3.18l) . 
Proof of (|3.17p . Using Lemma [TU1 equation (|2.2p one has for x £ Ms(x) fixed with 
y £ Pbx and z £ PaRbx 

\\x - x + \\ 2 
= 11^ — 2/1 1 2 

= || z — Rbx + Rbx — y\\ 2 



\z — Rbx\\ 



\Rbx - y\\ 



[3~T5l 2 2 

> ||z-i? s a;|| +\\R B x-y\\ 



-2(z — Rbx, Rbx — y ) 

£-N A (z) =y-xe-N B (y) 

2c \\z - R B x\\ \\Rbx - y\\ 



=(l-c) 



z — Rbx\ 



\Rbx - y\\ 



\z - R B x\\ 2 - 2 1| 2 - R B x\\ \\Rbx - y\\ + \\Rbx - y\\ 2 



= (1 



\\z - R B x\\ 



\Rbx - y\\ 



c [\\z - R B x\\ - \\Rbx - y\\Y 



>(l-c) \\z-R B x\\ 



\Rbx - y\\ 



(3.19) 



A 



Proof of (|3.18p . First note that if x £ Mg(x), since B is a subspace, by equation 
p.!2p R B x C Ba(x) and by convexity of Mg(x) it follows that y = Pbx C Ms(x) 
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and hence (|3.15[) is localized to Ms(x) (to which the dividends of linear regularity of 
the intersection extend) as long as P a Rbx C M§(x), that is, as long as x G U. By 
definition of the projector \\R B x — y\\ > \\RbX — Pb(Rbx)\\. Local linear regularity 
at x with radius S and modulus k yields 

\\z - R B x\\ 2 + \\R B x - y\\ 2 > \\z - R B x\\ 2 + \\R B x - P B R B x\\ 2 

=d (R B x, A) 2 + d (R B x, B) 2 

> —d(R B x,S) 2 V R B x G MsCx) (3.20) 

K 

This completes the proof of ()3.18[) and the Theorem. □ 

Remark 39. In (|3.19[) the difference [\\z — R B x\\ — \\R B x — y\\] is completely ignored. 
On the other hand we throw away either \\z — R B x\\ or \\R B x — P B R B x\\ in (|3.20[) . 

If p.!9p is tight then it is the worst possible result in (|3.20p . since \\z — R B x\\ = 
\\R B x — P B R B x\\. On the other hand if p. 201) is tight this means \\z — R B x\\ = or 
\\Rb% — PbRb%\\ — and this means that (I3.19|) is not tight. This shows that our 
estimation can never be tight. 

Lemma [35] with the added assumption of (e, J)-regularity of the nonconvex set 
yields local linear convergence of the AAR algorithm in this special case. 

Theorem 40. Let the closed subsets A, B o/E have locally linearly regular intersection 
at x G S = An B with regularity modulus k > and radius of regularity S in (|3.7D . 
Suppose further that B is a subspace and that A is (s , S) -regular at x. Assume that 
for some constant c G (0,1) the following condition holds: 



z e AnMs(x), ueN A (z)n) 
yeBnm s (x), veN B (y)n: 



(u,v)>-c. (3.21) 



If x G (P^x) r\M_s_(x) then 



d{x+,S) < ^l + e-n d(x,S) V x+ G T AAR x (3.22) 
with rj := and e = 2e + 2e 2 . 

Proof. As in the proof of (|3.18[) . if x G B i (x). since B is a subspace Rbx and 

l + e 

P B x G M_s_(x). Moreover, since A is (S, e)-regular at x, (|2 . 1 1 1) yields 



\\z-x\\ < (l + e)\\R B x-x\\ < (l + e) (^T^J Vz e P A R B x, V R B x G M s (x) 
and therefore P a Rbx C Ms(x). So the set U — M_s_(x) in A3. 16[) . In particular, on 

l + e 

(Pg x x) n B_i_ (x) we have 



— ^+11 > -d {x, S) = \\x — x\\ 

K K 

so the coercivity condition (|3.1j) is satisfied on (P^ 1 ^) n B s (x) 
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Now since A is (e, £)-regular and B is (0, oo)-regular, by Theorem [221 Ta a r is 
({x}, e)-firmly nonexpansive with e = 2e(l + e), that is 

at x E S, \\x + — x\\ 2 + \\x — x + \\ 2 < (1 + e) \\x — x\\ 2 V x + E Taarx, Vx E Mg(x). 
Lemma [2U1 then applies to yield 

||a; + - x\\ < + e -r)\\x-x\\ Vi + £ T AAR x, Vie (P s %) n B_i (x) 

where 77 = ^f. The observation that d(x + ,S) < \\x + —x\\ for all x + g Taarx 
completes the proof. □ 

The next lemma establishes sufficient conditions under which (|3.21[) holds. 

Lemma 41 ( [24] Theorem 5.16). Assume B C E is a subspace and that A C E is 
closed and super-regular at x E An B. If the intersection is strongly linearly regular 
at x, then there is a S > and a constant c E (0, 1) such that (j3.21l) holds. 

Proof. Condition (|3.15p can be shown using 13.61 For more details see [21] . □ 

We summarize this discussion with the following convergence result for the AAR 
algorithm in the case of an affine subspace and a super- regular set. 

Theorem 42. Assume B C E is a subspace and that A C E is closed and super- 
regular atxES = AC\B. If the intersection is strongly linearly regular at S, then 
there is a 8 > such that, ^ > 2e + 2e 2 and hence 

d(x+, S) < c d (x, S) \/x+eTaarx, (3.23) 



with c = 1 + 2e + 2e 2 - < 1 for all x E (P^x) H (x). 

Example 43. The example Example [5l (jvj) has been studied by Borwein and coauthors 
[TlfTU] where they achieve global characterizations of convergence with rates. Our work 
does not directly overlap with [UdU] since our results are local, and the order of the 
reflectors is reversed: we must reflect first across the subspace, then reflect across the 
nonconvex set; Borwein and coauthors reflect first across the circle. 

3.4 AAR on Subspaces 

We finish this section with the surprising fact that strong regularity of the intersection 
is necessary, not just sufficient for convergence of the AAR algorithm. 

Corollary 44. Let A, B be two affine subspaces with A B. AAR converges to a 
point in Af) B for all xq E E with linear rate c < 1 if and only if the intersection 
An B is strongly regular. 

Proof. If the intersection is strongly regular the requirements of Theorem 00] are glob- 
ally fulfilled for A, B subspaces, so AAR converges with linear rate 
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On the other hand for x G A n B by [6, Thm 3.5] we get get the characterization 

Fix Taar. =(AnB) + N A - B (0) 

= (AnB) + {N A (x)n~N B (x)). 

and so the fix point set of Taar does not coincide with the intersection if this is not 
strongly regular. □ 

Remark 45 (Friedrichs angle [IS])- The Friedrichs angle being less than 1 is not suffi- 
cient for convergence of AAR. This can be seen by example 131 (pi). A detailed analysis 
regarding the relation between the Friedrichs angle and linear convergence of MAP 
can be found in [T3"lll4| . 



4 Concluding Remarks 

We state the following theorem to suggest that the framework presented in this work 
can be extended to a more general setting, for example the adaptive framework dis- 
cussed in [7]. It shows that the (S 1 , e)-firm nonexpansiveness is preserved under convex 
combination of operators. 

Theorem 46. Let T\ be (S, E\)-fi,rmly nonexpansive and T 2 be (S, £i)-fi rmly nonex- 
pansive on D. The convex combination AT\ + (1 — \)T 2 is (S,e)-firmly nonexpansive 
on D where e = max{ei,£2}- 

Proof. Let x,y g D. Let 



x + eXTiX + (1 — A) T 2 x, 
y + e\T iy +(l-\)T 2 y, 

x+ =A.t ( | 1) + (l-\)x { +\ 

y+ =\yP + (l-\)y?, 



and 



where at+ £ T\x, x^ e T 2 x 
where y^ <E T x y, y^ € T 2 y. 



By Lemma [TjJ] (JTTJ) one has nonexpansiveness of the mappings given by 2T\x — x and 
2T 2 x — x, x £ D that is 



2x < i ) - x 



2x { + ) - x 



2yi 1} - y 
2yi 2) - y 



<Vl + 2ei||a;-j/|| , 



<V1 + 2e 2 \\x - y\\ 



This implies 
\\{2x 
( 2 



x)-(2y + -y)\\ 



XxV + (1 - X)x 



(2) 



AyW + (1 - \)y 



(2) 



2x { l ] - x 



2x { l ] - x 



2y i + ) - y 
2jA 1) - y 



-(l-A) 
+ (1 - A) 



<A 

<Vl + 2e||a;-y|| . 
Now using Lemma [T3] (Q) the proof is complete 



2xi 2) - x 



2x^ ] - x 



*vf - y 
2y ( + ] - y 



□ 
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Since the modulus of linear regularity does not recover optimal convergence results 
(see Remark I29[) . the question remains whether there is a quantitative primal defini- 
tion of a angle between two sets that recovers the same results for MAP as [5]. This 
could also be useful to achieve optimal linear convergence results for AAR. Another di- 
rection for further development is the metric regularity perspective briefly discussed in 
Proposition [331 This, we believe, will be helpful in extending the current applications 
beyond projection-type algorithms. 
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